Search for: All records

This article presents the Smart Quad, a tangible user interface (TUI) designed to enhance quadrilateral learning by integrating physical and digital educational tools. Developed through an iterative, user-centered design process, the Smart Quad is a tangible device that pairs with the PhET SimulationQuadrilateralcreating a multimodal, inclusive learning environment for geometry. We conducted a pilot study with four users with blindness or low vision (BLV) and a formal user study in two settings: a classroom study with 15 students from grades 6 to 8 and individual sessions with five students with BLV from grades 7 to 9. Our findings demonstrate the efficacy of the Smart Quad in facilitating hands-on, interactive learning experiences, particularly for students with BLV, and highlight the potential for TUIs to bridge gaps in mathematics education by supporting diverse learning needs and preferences. The results suggest that TUIs like the Smart Quad can significantly improve engagement and understanding of geometric concepts, offering a promising direction for future educational tools. 

Physical computing enables learners to create interactive projects using tangible materials and electronic components. These projects commonly utilize microcontroller boards like the micro:bit. In contrast, computer vision (CV) is a powerful technique for detecting input through interaction with everyday materials like paper, and it can be utilized for physical computing projects. However, CV-based toolkits are typically limited to input detection and rely on screen-based or projected outputs. This paper presents a hybrid approach that integrates a CV-based platform called Paper Playground with the micro:bit electronics platform. By combining CV-detected, paper-based inputs with the rich input-output possibilities of microcontroller-based systems, we showcase a multimodal physical computing toolkit. Through three project examples, we explore how this hybrid approach can enhance the creative possibilities in physical computing, and develop a preliminary design space combining CV-based and electronics-based physical computing. 

Paper prototyping presents a low-entry barrier method to engaging youth in interaction design. Purely paper-based designs leave a large gap between ideation and implementation. Paper Playground is a prototyping tool that connects physical and virtual papers with JavaScript programs, enabling the creation of multimodal prototypes in both face-to-face and virtual settings. Paper Playground is being designed and developed through iterative co-design activities including youth and adults. Here we present findings from remote co-design sessions with youth, investigating what affordances the participants requested from a multimodal prototyping tool. We reflect on the co-designers desires and remarks on paper use for interactive project design, remote collaborative use, and extensibility for physical computing. 

The evolution of Web Speech has increased the ease of development and public availability of auditory description without the use of screen reader software, broadening its exposure to users who may benefit from spoken descriptions. Building off an existing design framework for auditory description of interactive web media, we have designed an optional Voicing feature instantiated in two PhET Interactive Simulations regularly used by students and educators globally. We surveyed over 2000 educators to investigate their perceptions and preferences of the Web Speech-based Voicing feature and its broad appeal and effectiveness for teaching and learning. We find a general approval by educators of the Voicing feature and more moderate statement ratings than expected to the different preset speech levels we presented to them. We find that educators perceive the feature as beneficial both broadly and for specific populations while some acknowledge particular populations for whom it remains ineffective. Lastly, we identify some variance in the perceptions of the feature based on different aspects of the simulation experience. 

Auditory description display is verbalized text typically used to describe live, recorded, or graphical displays to support access for people who are blind or visually impaired. Significant prior research has resulted in guidelines for auditory description for non-interactive or minimally interactive contexts. A lack of auditory description for complex interactive environments remains a tremendous barrier to access for people with visual impairments. In this work, we present a systematic design framework for designing auditory description within complex interactive environments. We illustrate how modular descriptions aligned with this framework can result in an interactive storytelling experience constructed through user interactions. This framework has been used in a set of published and widely used interactive science simulations, and in its generalized form could be applied to a variety of contexts.